Papers with mixture-of-experts layers
Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that language group specialization on experts improves multilingual performance. |
| Approach: | They propose to dynamically group and scale up parameters of multilingual Large Language Models while boosting positive transfer among similar languages. |
| Outcome: | The proposed method reduces negative transfer between languages and boosts performance on 18 to 128 languages. |